The module routed C++ allocation to ValkeyModule_Alloc by replacing global
operator new/delete, and C allocation by #define-ing malloc and friends to
__wrap_* in the translation units that happened to include
memory_allocation_overrides.h. The macro layer was order-dependent -- the
header had to be included last or it mangled the malloc symbols in the
dependencies -- and covered only the TUs that opted in. Vendored C code
that called malloc directly, ICU and hdrhistogram, was simply exempt, and
its memory was never accounted for.

Define malloc, free, calloc, realloc, aligned_alloc, posix_memalign,
valloc and malloc_usable_size in the module instead. Everything linked
into it -- libstdc++'s operator new, abseil, protobuf, gRPC, ICU,
hdrhistogram and rax -- reaches them without opting in to anything.

This only works because of how the module is linked. A dlopened library's
symbol lookups search the global scope first, and libc.so.6 defines
malloc, so a module-defined malloc with default visibility is ignored --
even by the module's own operator new. versionscript.lds therefore marks
these symbols local: they are not exported, and being non-preemptible,
every reference inside the module binds to them at link time. The
-static-libstdc++ added earlier is the other half, putting operator new
inside the module in the first place.

Because the definitions are local, libc.so.6 never sees them and keeps
using its own allocator, so memory allocated and freed inside libc stays
self-consistent. The one way a pointer crosses is a libc function that
allocates a result and hands it back, which we would later release
through our free(), passing a libc pointer to ValkeyModule_Free. The
module references three: strdup, reached from absl::InitializeSymbolizer
and libstdc++'s message catalogs, is reimplemented here; realpath and
getcwd, reachable only from std::filesystem::canonical and current_path
which the module never calls, abort with a CHECK if that ever changes.
ICU's uprv_tzname uses the fortified __realpath_chk with a caller-
provided buffer, which does not allocate and is deliberately left alone.

The allocator is linked into the module only, not into vmsdklib: defining
malloc in a test executable would override the allocator for the whole
test process, including libc's own startup allocations. Unit tests
therefore run entirely on the system allocator, which is why
RaxMallocMemoryTracking becomes RaxAllocSizeReporting -- the accounting
it asserted on no longer exists in a test binary, and is enforced at link
time by ci/check_module_allocators.sh instead.

That script grows two checks, both of which caught real bugs in this
change while being written:

  - The module must define each allocator with local binding. Checking
    only that the symbol is defined is not enough: a build that omits the
    version script defines it globally, so it is preempted by libc and
    silently bypassed. Nothing crashes -- the module just runs on the
    system allocator with no accounting -- so only this check finds it.

  - No unhandled allocate-and-return libc function may be referenced. The
    watch list covers glibc's internal aliases, because <stdio.h> turns
    getline() into __getdelim() and the public name never appears.

versionscript.lds also gains a LINK_DEPENDS entry; without it, editing it
did not trigger a relink.

Verified: 22/22 unit tests in release, ASAN and TSAN, the latter with no
races; 369 passed and 8 skipped in the integration suite; clean module
load both standalone and with valkey-json loaded first; searches serve
correctly and accounting now tracks rax, growing 675KB to 3.18MB across
2200 indexed documents. Sanitizer builds compile the allocator out so the
sanitizer's own interceptors see every allocation.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BENZu4GYtPBoLAcKVVTHvc
Signed-off-by: Allen Samuels <allenss@amazon.com>

Nothing in the module allocates before ValkeyModule_Alloc is established:
static initializers are deferred until after it is set, and the counters
that watched for a violation never once fired. Carrying that check meant
every allocator kept a branch on an IsUsingValkeyAlloc() flag, a fallback
path into __libc_malloc, and accounting indirected through function
pointers so the same helper could serve both allocators.

Drop the detection and the fallback. If something ever does allocate too
early, ValkeyModule_Alloc is still unset and the call faults at the
offending call site; Valkey's crash handler prints the backtrace and
addr2line resolves the frame to the exact line. That localises the
problem better than the counter did, which recorded a single return
address and reported it later, from somewhere else.

With nothing left to select between two allocators, the whole switch --
IsUsingValkeyAlloc, UseValkeyAlloc and ResetValkeyAlloc, which had no
callers of its own -- becomes dead, and the four PerformAndTrack helpers
collapse into the allocators they served. The accounting now inlines at
each call site instead of going through a function pointer.

ResetValkeyAllocStats goes too; its only caller was ResetValkeyAlloc.
vmsdk/src/memory_allocation.h is left smaller than it was before any of
this work, holding just the six accounting functions that have callers.

Verified: 22/22 unit tests in release, ASAN and TSAN, the latter with no
races; 369 passed and 8 skipped in the integration suite; clean module
load with valkey-json loaded first, searches served, and 500 documents
indexed with the memory accounted. The fault behaviour was confirmed by
temporarily running the deferred initializers ahead of ValkeyModule_Init,
which crashed with the backtrace naming memory_allocation_c_api.cc:83.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BENZu4GYtPBoLAcKVVTHvc
Signed-off-by: Allen Samuels <allenss@amazon.com>

It looks removable -- a bespoke allocator whose only users all pass the
tag that disables its reporting -- so record what happens if you try.

The accounting counters are themselves ShardedAtomics, so
ReportAllocMemorySize -> ShardedAtomic::Add allocates: it constructs a
thread_local ThreadLocalNode, registers it in a vector, and grows the
node's value array under resize_mutex. Routing those allocations through
the module allocator makes each one call ReportAllocMemorySize again,
re-entering either a thread_local's own initialization or a non-reentrant
absl::Mutex. Substituting std::allocator hangs the module during load on
a futex, accumulating no CPU time, before the server accepts connections.

Unit tests do not catch this: they never link the module allocator, so
there std::allocator reaches glibc and nothing reports. It only fails in
the module.

Allocating from Valkey while skipping the accounting would break the
cycle as well, but ShardedAtomic is linked into the test executables too,
where ValkeyModule_Alloc is a mock that stays unset until a fixture
installs it. Going straight to glibc is what keeps this allocator
independent of everything it underpins.

Comment only; no functional change.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BENZu4GYtPBoLAcKVVTHvc
Signed-off-by: Allen Samuels <allenss@amazon.com>

Every DSO the module links against is another allocator boundary. Memory
one of them allocates and hands back belongs to its allocator, not ours,
and releasing it here would pass it to ValkeyModule_Free. Check 5 already
covers libc, the only such library whose allocate-and-return functions the
module calls directly, but nothing noticed a new dependency arriving.

Pin the set instead, so an unreviewed boundary fails the build. Only
additions fail; a dependency disappearing is not a memory-safety problem,
and is how this list last changed, when -static-libstdc++ removed
libstdc++.so.6.

The existing non-libc dependencies were reviewed while writing this, and
the findings are recorded next to the check so nobody has to repeat the
work. libsystemd contributes four socket predicates that return int.
OpenSSL's imported constructors and duplicators are each paired with the
matching free function, also imported; its raw-buffer cases
(ASN1_STRING_to_UTF8, the i2d_* family with a null output pointer) are
released by OPENSSL_free, which is CRYPTO_free inside libcrypto, so both
the allocation and the free happen on the far side of the boundary, as
with getaddrinfo/freeaddrinfo. libm, libmvec, libgcc_s and the dynamic
loader allocate nothing we free.

OpenSSL stays dynamically linked on purpose: static linking would mean
rebuilding the module for every OpenSSL CVE instead of picking up a
distribution update.

Verified both ways: the check passes on the module and reports
libstdc++.so.6 against valkey-json's module, which links it dynamically.
It also caught a bug in its own first draft, where the multi-line
allowlist left libgcc_s.so.1 followed by a newline rather than a space
and so never matched.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YSbN5Z97cb3ZRtH9kJpCHS
Signed-off-by: Allen Samuels <allenss@amazon.com>

Two constructs in this branch are glibc/ELF-only and broke the macOS job.

Mach-O has no weak undefined symbol. On ELF a weak declaration of a
missing symbol resolves to a null address, which is how
RunDeferredStaticInitializers detects a build where the linker script did
not relocate .init_array. Mach-O has no equivalent: the declaration is
simply an unresolved reference, and the link fails with

  Undefined symbols for architecture arm64:
    "___vmsdk_init_array_end", referenced from:
        vmsdk::RunDeferredStaticInitializers() in libvmsdklib.a

Static initialization is never deferred on macOS anyway -- the relocation
is done by a GNU linker script -- so declare the bounds symbols only where
they can exist and return 0 there.

__libc_malloc and __libc_free are glibc-only. RawSystemAllocator names
them so that the module's own malloc/free cannot capture it, but on builds
where the module defines no allocator -- sanitizer builds and macOS -- the
plain names already are the system allocator. Route through RawSystemMalloc
and RawSystemFree, which pick per build. This one had not failed yet: the
macOS link died at the first executable pulling in deferred_init.cc.o,
before reaching one that instantiates RawSystemAllocator.

Verified on Linux (22/22 unit tests, all guards pass) and by compiling both
files with the __APPLE__ path taken, which is as close as this machine gets
to the macOS build.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YSbN5Z97cb3ZRtH9kJpCHS
Signed-off-by: Allen Samuels <allenss@amazon.com>